GRADIENT-ENHANCED DEEP NEURAL NETWORK APPROXIMATIONS
نویسندگان
چکیده
We propose in this work the gradient-enhanced deep neural network (DNN) approach for function approximations and uncertainty quantification. More precisely, proposed adopts both evaluations associated gradient information to yield enhanced approximation accuracy. In particular, is included as a regularization term DNN approach, which we present posterior estimates (by two-layer networks) similar those path-norm regularized approximations. also discuss application of quantification, several numerical experiments show that can outperform traditional many cases interest.
منابع مشابه
Gradient-enhanced Neural Network Response Surface Approximations
An approach to develop response surface approximations based upon artificial neural networks trained using both state and sensitivity information is described in this paper. Compared to previous approaches, this approach does not require weighting the residuals of the targets and gradients and is able to approximate gradient-consistent response surfaces with a relatively compact network archite...
متن کاملAccelerating deep neural network training with inconsistent stochastic gradient descent
Stochastic Gradient Descent (SGD) updates Convolutional Neural Network (CNN) with a noisy gradient computed from a random batch, and each batch evenly updates the network once in an epoch. This model applies the same training effort to each batch, but it overlooks the fact that the gradient variance, induced by Sampling Bias and Intrinsic Image Difference, renders different training dynamics on...
متن کاملSpeeDO: Parallelizing Stochastic Gradient Descent for Deep Convolutional Neural Network
Convolutional Neural Networks (CNNs) have achieved breakthrough results on many machine learning tasks. However, training CNNs is computationally intensive. When the size of training data is large and the depth of CNNs is high, as typically required for attaining high classification accuracy, training a model can take days and even weeks. In this work, we propose SpeeDO (for Open DEEP learning ...
متن کاملA conjugate gradient based method for Decision Neural Network training
Decision Neural Network is a new approach for solving multi-objective decision-making problems based on artificial neural networks. Using inaccurate evaluation data, network training has improved and the number of educational data sets has decreased. The available training method is based on the gradient decent method (BP). One of its limitations is related to its convergence speed. Therefore,...
متن کاملLow-rank Representation for Enhanced Deep Neural Network Acoustic Models
Automatic speech recognition (ASR) is a fascinating area of research towards realizing humanmachine interactions. After more than 30 years of exploitation of Gaussian Mixture Models (GMMs), state-of-the-art systems currently rely on Deep Neural Network (DNN) to estimate class-conditional posterior probabilities. The posterior probabilities are used for acoustic modeling in hidden Markov models ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Journal of machine learning for modeling and computing
سال: 2022
ISSN: ['2689-3967', '2689-3975']
DOI: https://doi.org/10.1615/jmachlearnmodelcomput.2022046782